PANACEA: An Automated Misinformation Detection System on COVID-19

In this demo, we introduce a web-based misinformation detection system PANACEA on COVID-19 related claims, which has two modules, fact-checking and rumour detection. Our fact-checking module, which is supported by novel natural language inference methods with a self-attention network, outperforms state-of-the-art approaches. It is also able to give automated veracity assessment and ranked supporting evidence with the stance towards the claim to be checked. In addition, PANACEA adapts the bi-directional graph convolutional networks model, which is able to detect rumours based on comment networks of related tweets, instead of relying on the knowledge base. This rumour detection module assists by warning the users in the early stages when a knowledge base may not be available. © 2023 Association for Computational Linguistics.

Evaluating the generalisability of neural rumour verification models

Research on automated social media rumour verification, the task of identifying the veracity of questionable information circulating on social media, has yielded neural models achieving high performance, with accuracy scores that often exceed 90%. However, none of these studies focus on the real-world generalisability of the proposed approaches, that is whether the models perform well on datasets other than those on which they were initially trained and tested. In this work we aim to fill this gap by assessing the generalisability of top performing neural rumour verification models covering a range of different architectures from the perspectives of both topic and temporal robustness. For a more complete evaluation of generalisability, we collect and release COVID-RV, a novel dataset of Twitter conversations revolving around COVID-19 rumours. Unlike other existing COVID-19 datasets, our COVID-RV contains conversations around rumours that follow the format of prominent rumour verification benchmarks, while being different from them in terms of topic and time scale, thus allowing better assessment of the temporal robustness of the models. We evaluate model performance on COVID-RV and three popular rumour verification datasets to understand limitations and advantages of different model architectures, training datasets and evaluation scenarios. We find a dramatic drop in performance when testing models on a different dataset from that used for training. Further, we evaluate the ability of models to generalise in a few-shot learning setup, as well as when word embeddings are updated with the vocabulary of a new, unseen rumour. Drawing upon our experiments we discuss challenges and make recommendations for future research directions in addressing this important problem. © 2022 The Author(s)

Natural Language Inference with Self-Attention for Veracity Assessment of Pandemic Claims

We present a comprehensive work on automated veracity assessment from dataset creation to developing novel methods based on Natural Language Inference (NLI), focusing on misinformation related to the COVID-19 pandemic. We first describe the construction of the novel PANACEA dataset consisting of heterogeneous claims on COVID-19 and their respective information sources. The dataset construction includes work on retrieval techniques and similarity measurements to ensure a unique set of claims. We then propose novel techniques for automated veracity assessment based on Natural Language Inference including graph convolutional networks and attention based approaches. We have carried out experiments on evidence retrieval and veracity assessment on the dataset using the proposed techniques and found them competitive with SOTA methods, and provided a detailed discussion.

OHARS: Second workshop on online misinformation-and harm-aware recommender systems

Recommender systems play a central role in online information consumption and user decision-making by leveraging user-generated information at scale to assist users in finding relevant information and establishing new social relationships. Just as recommendation techniques have become powerful tools that are inserted in most social platforms, they could also involuntarily spread unwanted content and other types of online harms. The same fundamental concepts on which these techniques rely make them facilitators of such unwanted diffusion. To increase the user-perceived quality of recommender systems and mitigating the negative effects of the multiple forms of online harms, it is essential to provide recommender systems with harm-aware mechanisms. To further research in this direction, this Second edition of the Workshop on Online Misinformation-and Harm-Aware Recommender Systems (OHARS 2021) aimed at fostering research in recommender systems that can mitigate the negative effects of online harms by fostering the recommendation of safe content and trustworthy users, with a special interest in research tackling the negative effects of the propagation of harmful content referring to the COVID-19 crisis. © 2021 Owner/Author.